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e) projecting the grid on the object; the grid/being located 
at the first position; / 

f) taking with the camera an image ofjhe object illuminated 
by the projected grid; the image of the object haying intensity values for 

5 each pixel position; 

g) repeating steps e) and f) abfeast two times with the grid 
being located at the two different positions/to yield at least three intensity 
values for each pixel; 

h) computing the object^phase for each pixel position using 
1 0 the at least three object intensity values for the corresponding pixel; 

i) computing the difference of height between the object 
and the reference object for eac,h pixel using the reference object phase 
and the object phase for the corresponding pixel; and 

j) using the difference of heights between the object and 
1 5 the reference object for each pixel to determine the relief of the object. 



Jinq to^j 



According to a another aspect of the present invention, there 
is provided a method ^or measuring the height of a module including at 
least one object mounted on a generally plane substrate using a camera 
20 provided with an array of pixels, the method comprising: 

a) projecting a grid onto the module; the grid being located 
at a first position relative to the camera and to the module; 
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b) taking, with the camera, an image oy the module 
illuminated by the projected grid; the image of the modulp4iaving intensity 
values for each pixel; 

c) repeating steps a) and b) at least two times with the grid 
5 being located at two different known positions relative to the camera and 

to the module to yield at least three intensity values for each pixel; 

d) computing the module phase^or each pixel using the at 
least three module intensity values for the corresponding pixel; 

e) computing a complementary phase of the substrate by 
10 using the at least three intensity values/from pixels on the image of the 

module not corresponding to the at least one object; 

f) computing the height of the at least one object for each 
pixel using the complementary phase of the substrate and the module 
phase for the corresponding pixel/ 

15 g) projecting the gjid onto a reference plane; the grid being 

located at the first position; 

h) taking with thfe camera an image of the reference plane 
illuminated by the projected grid; the image of the reference plane having 

intensity values for each pixel position; 

/ 

20 i) repeating^steps g) and h) at least two times with the grid 

being located at the two different positions to yield at least three intensity 



values for each pixel; 



/ 



j) computing the reference plane phase for each pixel 
position using the at least three reference plane intensity values for the 
25 corresponding pixel; 



1 
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k) computing the height of the substrate for each pixel 

using the complementary plane phase of the substrate anjj the reference 

plane phase for the corresponding pixel; and 

computing the height of the module by adding the height of 

/ 

5 the substrate and the height of the at least one object. 

According to another aspect of the/present invention, there 
is also provided a system for measuring the relief of an object, the system 
comprising: / 
10 a grid projecting assembly; the grid projecting assembly 

including a grid, an illuminating assembly tjpat includes a light source to be 
projected through the grid onto the object and a projector to project the 
grid onto the object; the grid being mounted to a support; 

an image acquisition apparatus including a camera provided 



ippar 



15 with an array of pixels; 

a computer configured for 

a) positioning the grid relative to the object and to the 
( image acquisition apparatus; j 

b) receiving from the image acquisition apparatus at least 

/ 

20 three images of the projected grid onto the object and at least three 

images of the projected grid onto a reference object; each of the images 

/ 

of the projected grid onto the object corresponding to a different known 
position of the grid; each of the images of the projected grid onto the 
reference object corresponding to one of the known positions of the grid; 
25 c) computing|the reference object phase for each pixel 

using the at least three reference object intensity values for the 
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corresponding pixel; 

d) computing the object phase for each pixel using the at 
least three object intensity values for the corresponding pixel; and 

e) computing the difference of height^between the object 
and the reference object for each pixel using the reference object phase 
and the object phase for the corresponding pixe 



Other objects, advantages and features of the present 
invention will become more apparent up^>n reading the following non- 
10 restrictive description of preferred embodiments thereof, given by way of 
example only, with reference to the accompanying drawing 



BRIEF DESCRIPTION OF THE DRAWINGS 



Figure 1 is a schematic view of a system for inspecting 
the surface of an object according to an embodiment of the present 



invention; 



Figure 2 is a schematic view of both the image 
acquisition apparatus and the grid projection assembly of Figure 1; 



25 
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WHAT IS CLAIMED IS: 

f 

1. A method for measuring the relief of an object using a 
camera provided with an array of pixels, said method comprising: 

a) projecting a grid on ayreference object; the grid being 
located at a first position relative to ,the camera and to the reference 



/ 



object; 

b) taking, with the camera, an image of the reference object 

illuminated by said projected grid; said image of the reference object 

/ 

10 having intensity values for each pixel; 

c) repeating steps a) and b) at least two times with the grid 
being located at two different known positions relative to the camera and 
to the reference object to yield at least three intensity values for each 
pixel; / 

15 d) computingihe reference object phase for each pixel using 

the at least three reference object intensity values for the corresponding 
pixel; 

e) projectjhg tne 9 nd on tne ob J ect i tne Qnd being located at 
said first position; 

20 f) taking with the camera an image of the object illuminated 

by said projected gr/d; said image of the object having intensity values for 
each pixel positior 

g) repeating steps e) and f) at least two times with the grid 
being located at said two different positions to yield at least three intensity 

25 values for each pixel; 

h) /computing the object phase for each pixel position using 
the at least three object intensity values for the corresponding pixel; 
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i) computing the difference/of height between the object 
and the reference object for each pixel using said reference object phase 
and said object phase for the corresponding pixel; and 

j) using said difference^ heights between the object and 
the reference object for each said/pixel to determine the relief of the 
object. 



2. A method as recited in claim 1, wherein, in at least one 
/ 

of steps d) and h), the phase &® is computed for each pixel by solving 
10 the following system of equations: 

J n z=\a + B • cos(A<D + A<p n ) 



15 



20 



25 



where l n represent the at least three intensity values, A and B are known 
coefficients and A<p n aje phase variations caused by the different 
locations of the grid. 



3. A method as recited in claim 1, wherein, in step c), steps 
a) and b) are repeated more than two times with the grid being located at 
more than two differentjknown positions relative to the camera and to the 
reference object to yield more than three intensity values and, in step d), 
only the three most advantageous values from said more than three 
intensity values are selected and used to compute the reference object 
phase for each pixel. 



4. A me 
a) and b) are repeated 
more than two differen 



hod as recited in claim 1, wherein, in step g), steps 
more than two times with the grid being located at 
known positions relative to the camera and to the 



V 
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object to yield more than three intensity values and, in step d) only the 
three most advantageous values form said/more than three intensity 
values are selected and used to compute the object phase for each pixel. 



5. A method as recited in/claim 1, wherein, in step c), said 
different known positions of the grid are chosen so as to provide in step 
g) at least two images of the object ^having a 180 degrees difference in 
phase therebetween. 



10 6. A method as' recited in claim 5, wherein a two- 

dimensional image of the objec/is computed by subtracting said at least 
two images of the object h^/ing a 180 degrees difference in phase 
therebetween; said two dimensional image being used to perform a 
preliminary analysis of the object. 



7. A method'as recited in claim 1, wherein, in step g), said 

different known positions of the grid are chosen so as to provide at least 

/ 

two images of the reference object having a 180 degrees difference in 
phase therebetween. 



8. A method as recited in claim 7 t wherein a two- 



dimensional image of the reference object is computed by subtracting said 
at least two images of the reference object having a 180 degrees 

difference in phase therebetween; said two dimensional image being used 

f 

25 to perform a preliminary analysis of the reference object. 



9. A method as recited in claim 1, wherein said reference 
object is a plane surface. 
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10. A method as recited in claim 1, wherein said reference 
object is said object at a past predetermined tinji^ and said reference 
object phase is computed around said past time; 

whereby step i) provides the variation of height^at each pixel between said 
past time and the approximate time when the object phase is computed; 
and step j) yield the variation with time of relief of the object. 



1 1 . A method as recited .of claim 1 , wherein said reference 
10 object is a CAD of the object; said^rid being virtually positioned and 
projected into said CAD in step a) apd said image of said reference object 
being simulated in step b). 



12. A method for measuring the height of a module including 
15 at least one object mounted^n a generally plane substrate using a 
camera provided with an array of pixels, said method comprising: 



ray 

I) projecting ajgrid onto the module; the grid being located 
at a first position relative to the camera and to the module; 

m) taking, with the camera, an image of the module 

20 . illuminated by said projected grid; said image of the module having 

I 

intensity values for each pixel; 

n) repeating steps a) and b) at least two times with the grid 

if 

being located at two different known positions relative to the camera and 
to the module to yield atjleast three intensity values for each pixel; 
25 o) computing the module phase for each pixel using the at 

least three module intensity values for the corresponding pixel; 

/ 

/ 
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p) computing a complementary phase of the substrate by 
using said at least three intensity values from pixels on the image of the 
module not corresponding to said at least one object; 

q) computing the height of said at least one object for each 
5 pixel using said complementary pha^e of the substrate and said module 
phase for the corresponding pixel; 

r) projecting the grip onto a reference plane; the grid being 
located at said first position; 

s) taking with th£ camera an image of the reference plane 
10 illuminated by said projected^ grid; said image of the reference plane 
having intensity values for each pixel position; 

t) repeating steps g) and h) at least two times with the grid 
being located at said two different positions to yield at least three intensity 
values for each pixel; / 
15 u) computing the reference plane phase for each pixel 

position using the at least three reference plane intensity values for the 
corresponding pixel; / 

v) computing the height of the substrate for each pixel 
using said complementary plane phase of the substrate and said 
20 reference plane phase for the corresponding pixel; and 

w) computing the height of the module by adding the height 
of the substrate and the height of said at least one object. 



13. 



25 inspection. 



14. 

system comprising 



"he use of the method of claim 12 for lead-coplanarity 



A system for measuring the relief of an object, said 
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a grid projecting assembly; said grid projecting assembly 
including a grid, an illuminating assembly that includes a light source to be 
projected through said grid onto the object and a projector to project the 
grid onto the object; said grid being mdunted to a support; 
5 an image acquisition apparatus including a camera provided 

with an array of pixels; 

a computer configured for 

a) positioning saip grid relative to said object and to said 
image acquisition apparatus; 
10 b) receiving froyh the image acquisition apparatus at least 

three images of the projected grid onto the object and at least three 
images of the projected grid c^nto a reference object; each of said images 
of the projected grid onto the object corresponding to a different known 
position of the grid; each of said images of the projected grid onto the 
1 5 reference object corresponding to one of said known positions of the grid; 

c) computing the reference object phase for each pixel 
using the at least three reference object intensity values for the 
corresponding pixel; 
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d) computing the object^)hase for each pixel using the at 
least three object intensity values fop4he corresponding pixel; and 

e) computing the difference of height between the object 
and the reference object for eacl^/pixel using said reference object phase 
and said object phase for the corresponding pixel. 
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I. Basis of the report 

1 . With regard to the elements of the international application (Replacement sheets which have been furnished to 
the receiving Office in response to an invitation under Article 14 are referred to in this report as "originally filed" 
and are not annexed to this report since they do not contain amendments (Rules 70. 16 and 70.17)}. 
Description, pages: 

1 -4,7-26 as originally filed 

5.6.6A-6B as received on 17/05/2001 with letter of 17/05/2001 
Claims, No.: 

V14 as received on 17/05/2001 with letter of 17/05/2001 
Drawings, sheets: 

1/1 5-1 5/1 5 as originally filed 



2. With regard to the language, all the elements marked above were available or furnished to this Authority in the 
language in which the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1 (b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under Rule 
55.2 and/or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 
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□ the claims, Nos.: 

□ the drawings, sheets: 

5. □ This report has been established as if (some of) the amendments had not been made, since they have been 
considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 



6. Additional observations, if necessary: 



V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1. Statement 



Novelty (N) 


Yes: 


Claims 


3-8,10-14 




No: 


Claims 


1,2,9 


Inventive step (IS) 


Yes: 


Claims 


3,4,11-13 




No: 


Claims 


1,2,5-10,14 


Industrial applicability (IA) 


Yes: 


Claims 


1-14 




No: 


Claims 





2. Citations and explanations 
see separate sheet 

VII. Certain defects in the international application 

The following defects in the form or contents of the international application have been noted: 
see separate sheet 

VIII. Certain observations on the international application 

The following observations on the clarity of the claims, description, and drawings or on the question whether the 
claims are fully supported by the description, are made: 
see separate sheet 



Form PCT/IPEA/409 (Boxes l-VIII, Sheet 2) (July 1998) 




INTERNATIONAL PRELIMINARY International application No. PCT/CA00/00834 
EXAMINATION REPORT - SEPARATE SHEET 



Re Item V (novelty, inventive step, industrial applicability) 

Reference is made to the following documents of the search report: 

D1: EP-A-0 182 469 (NEW YORK INST TECHN) 28 May 1986 (1986-05-28) 
D2: WO 98 55826 A (ELECTRONIC PACKAGING SERVICES ;WANG YINYAN 

(US)) 10 December 1998 (1998-12-10) 
D3: WO 88 02847 A (EASTMAN KODAK CO) 21 April 1 988 (1 988-04-21 ) 

• Negative statements 

1. Lack of novelty (Article 33(2) PCT) 

1.1. The independent claim 1 is not new over document D1 which discloses ( the reference in 
parenthesis referring to D1 ): 

A method for measuring the relief of an object (cf. title and abstract; Figure 7) using a 
camera (detector, 120; claim 1) provided with an array of pixels (image sensing array; page 
12, first paragraph), said method comprising (cf. page 5, paragraphs 1 and 2; page 21, last 
paragraph; Figures 4A, 4B, 5, 8A, and 8B; page 23, second paragraph): 
(a) projecting a grid on a reference object (reference plane); the grid being located at a first 
position relative to the camera and to the reference object; (b) taking, with the camera 
(linear array camera, 1420), an image of the reference object illuminated by said projected 
grid; said image of the reference object having intensity values for each pixel; repeating 
steps (a) and (b) at least two times (three frames each of reference plane; translation of 
grating, cf. page 25 first paragraph; Figure 8A) with the grid being located at two different 
known positions relative to the camera and to the reference object to yield at least three 
intensity values for each pixel; (d) computing the reference object phase for each pixel 
using the at least three reference object intensity values for the corresponding pixel (Figure 
8A); (e) projecting the grid on the object; the grid being located at said first position; (f) 
taking with the camera an image of the object (implicit) by said projected grid; said image 
of the object having intensity values for each pixel position (implicit); (g) repeating steps (e) 
and (f) at least two times with the grid being located at said two different positions to yield 
at least three intensity values for each pixel (Figure 8A); (h) computing the object phase for 
each pixel position using the at least three object intensity values for the corresponding 
pixel (Figure 8A); (i) computing the difference of height between the object and the 
reference object for each pixel using said reference object phase and said object phase for 
the corresponding pixel (Figures 4B and 8B; pages 25-26); and (j) using the difference of 
heights between the object and the reference object to detremine the relief of the object 
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(eg. Figure 10 shows an object's relief). 

1 .2. The dependent claims 2 and 9 are not new over document D1 : 

Claim 2: in D1 , the same system of intensity value equations (cf. D1 : page 25) as used in 
the present application is solved for each pixel of the images. In D2, the same set of 
equations (cf. D2: abstract, equations on page 6) are also used. 

Claim 9: a reference plane is also used in D1 as reference object (cf. D1 : page 21 , last 
paragraph; Figure 5). 



2. Lack of inven tive step (Article 33(3) PCT) 

The following claims do not involve an inventive step, because the person skilled in the art 
would arrive at the subject-matter of those claims by combining the teaching from closest 
prior art document D1 with teaching of documents D2 or D3: 

2.1 . Independent claim 14: 

Document D1 discloses (the references in parenthesis referring to the document D1): 

A system for measuring the relief of an object (Figures 1 and 14), the system comprising: 

• a grid projecting assembly (grating projector, 1410), including a grid (G; Figure 7), an 
illuminating assembly including a light source (eg. laser, 111; Figure 2) to be projected 
through the grid onto the object, a projector to project the grid onto the object; 

• an image acquisition apparatus including a camera provided with an array of pixels (linear 
array camera, 1 420); 

• a computer (processor, 150; digital computer associated with memory and peripherals; 
page 12, first paragraph) configured for 

- receiving images of the projected grid on a reference object from the image acquisition 
apparatus; 

- computing the reference object phase for each pixel using the at least three reference 
object intensity values for the corresponding pixel (Figure 4A; page 16, second paragraph; 

- computing the object phase for each pixel using the at least three object intensity values 
for the corresponding pixel (Figure 4A); and 

- computing the difference of height between the object and the reference object for each 
pixel using said reference object phase and said object phase for the corresponding pixel 
(Figure 4B). 
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The difference between the subject-matter of claim 14 and D1 is that in claim 14, the grid is 
mounted on a support and that the computer is configured for positioning the grid relative 
to the object and the image acquisition system. 

The problem to be solved could therefore be regarded as how to change the relative 
position of the surface to be measured with respect to the grid. 

In D1 , for instance, the problem is solved in that the object is mounted on a rotary stage 
(1490; Figure 13A), the movement of the rotary stage being controlled by a processor 
(1470, Figure 14) which allows to change the position of the object's surface relative to the 
grid and the acquisition system. If confronted with the problem, the person skilled in the art 
would, without involving an inventive step, consider either to change the position of the grid 
or that of the object to be measured. Furthermore, controlling the movements of a system's 
component, in this case the grid mounted on a support, with a computer lies within the 
general trend of the technique. 

2.2 Dependent claims 5 and 6 relating to the reference object, and claims 7 and 8 relating 
to the object: 

In the interferometry method described in D2, the Moire fringe patterns are computed using 
the Light Intensity Equations (cf. D2, page 6, lines 12-14) with different phase values, two 
of which have 1 80 degrees, ie. n rad, difference. The relative position of the 
object/reference object is sequentially chosen as to create four phase-shifted Moire fringe 
patterns (cf . D2: page 6, lines 3-1 1 ). The thus obtained intensity images are then 
subtracted (cf. D2: equation page 7). The result is then used for further data processing 
steps, eg. filtering smoothing, as described in D2 to provide preliminary analysis of the 
object/reference object. 

2.3. Dependent claim 10: 

The embodiment on page 20 of D2 also describes an interferometry method to determine 
the variation of the object's surface with temperature over time. For that purpose, the 
measurements are repeated to determine the change in surface flatness, ie. variations of 
heights, of the object (cf. D2: page 20, lines 6-14). 

2.4. Claims 5, 6, 7, 8, 10, and 14 do therefore not involve an inventive step (Article 33(3) PCT). 
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• Positive statements 

3.1 . The closest prior art document D1 shows a method for measuring the relief of an object as 
cited under paragraph 1.1 . above. In the independent method claim 11, which refers to 
claim 1 , such a method is claimed, however with the reference object being replaced by a 
CAD of the object. 

The subject-matter of claim 1 1 differs from that of D1 in that: 

• a CAD of the object is used instead of a reference object; 

• the grid is virtually positioned and projected into said CAD; and 

• the reference object is simulated. 

This method allows an easier control of the positioning of the grid with respect to the 
reference object, ie. the CAD, thus improving the accuracy of the measurements. 

In D3, a CAD of the object is employed only to find a starting location for the image 
reconstruction process, whereas in the present application, the CAD of the object replaces 
the reference object. Although most technical features are known from documents D1 , D2 
and D3, the prior art does not teach nor suggest to combine those features and. use them 
in an interferometry method and for the purposes as claimed. 

3.2. The closest prior art document D1 shows a method for measuring the relief of an object as 
cited under paragraph 1 .1 . above. The interferometry method defined in the independent 
claim 12 is used in particular for measuring the height of a module consisting of at least 
one object mounted on a substrate. 

The subject-matter of claim 12 differs from that of D1 in that: 

• a complementary phase of the substrate is measured and subsequently is used to 
compute the height of the object; 

• the height of the substrate for each pixel is computed using the complementary phase of 
the substrate and the phase of a reference plane for the corresponding pixel; and 

• the height of the module is computed by adding the height of the substrate and the 
height of the object. 

The method, which employs an approximation of the surface of the substrate as a 
complementary reference object, allows to measure the height of a module, especially 
when the substrate is not parallel to a plane surface, with a high accuracy without 
increasing the computation time. 
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The prior art does not teach nor suggest to compute an approximation of the surface of the 
substrate as a second reference object in order to derive the height of a module in a 
manner as claimed. 

3.3. The independent claim 13 refers to the use of the method defined in claim 12 for lead- 
coplanarity inspectoin. Claim 12 being novel and inventive, claim 13 is also novel and 
inventive in the sense of Article 33(2) and (3) PCT. 



3.4. In claims 3 and 4 depending of claim 1 , it is defined that additional images of the reference 
object and the object, respectively, are taken to allow for the selection of the most 
advantageous intensity values for a particular pixel. In prior art, however, taking additional 
images has the purpose of increasing the precision of the phase computation by using all 
values in higher order computations. 

3.5. Claims 3, 4, and 11-13 are therefore novel and involve an inventive step in the sense of 
Article 33(2) and (3) PCT. 

4. All claims are doubtless industrally applicable (Article 33(4) PCT). 



Re Item VII (Certain defects) 

1. Independent claims 1,12, and 14 are not in the two-part form in accordance with Rule 
6.3(b) PCT, which in the present case would be appropriate, with those features known in 
combination from the prior art (document D1 ) being placed in the preamble (Rule 6.3(b)(i) 
PCT) and with the remaining features being included in the characterising part (Rule 
6.3(b)(ii) PCT). 

2. The features of the claims are not provided with reference signs placed in parentheses 
(Rule 6.2(b) PCT). 

3. The documents D1 , D2, and D3 have not been identified in the description and the relevant 
background art briefly discussed. A document reflecting the prior art described on pages 
1-4 is not identified in the description (Rule 5.1(a)(ii) PCT). 
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Re Item Vlll (Certain observations) 

1 . Lack of clarity of claims (Article 6 PCT): 

1.1. Claim 1 : In the method steps (c) and (g), it is unclear whether during the repetition of the 
steps (a) and (b), and (e) and (f), respectively, the grid is being located at at least two 
different known position relative to the camera and the reference object or not. 

1 .2. Claim 5 and claim 6, and 7 and 8: Ambiguity arises due to the fact that claims 5 and 6 
refer to step (c) of claim 1 , which deals with the reference object , whereas claims 5 and 6 
refer to the object . The same applies to claims 7 and 8 dealing with the reference object, 
for their reference to the object of step (g) of claim 1 . 

2. Contradiction between claims and description (Article 6 PCT): 

2.1. Claims 6 and 8: In the description (page 19, line 13) the two dimensional image of the 
object and the reference object, respectively, is inferred by adding the at least two images 
having 180 degrees difference in phases therebetween. In claims 6 and 8, however, a 
substraction (page 28, line 11 and line 22) is claimed. 

2.2. The vague and imprecise statement "...spirit of the invention..." in the description on page 
pages 9 ( line 1 6), 1 3 ( line 21 ), 22 (line 1 8), 26 (line 8) implies that the subject-matter for 
which protection is sought may be different to that defined by the claims, thereby resulting 
in lack of clarity when used to interpret them. 
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e) projecting the grid on the object the grid being located 
at the first position; 

f) taking with the camera an image of the object illuminated 
by the projected grid; the image of the object having intensity values for 

5 each pixel position; 

g) repeating steps e) and f) at least two times with the grid 
being located at the two different positions to yield at least three intensity 
values for each pixel; 

h) computing the object phase for each pixel position using 
1 0 the at least three object intensity values for the corresponding pixel; 

i) computing the difference of height between the object 
and the reference object for each pixel using the reference object phase 
and the object phase for the corresponding pixel; and 

j) using the difference of heights between the object and 
1 5 the reference object for each pixel to determine the relief of the object 

According to a another aspect of the present invention, there 
is provided a method for measuring the height of a module including at 
least one object mounted on a generally plane substrate using a camera 
20 provided with an array of pixels, the method comprising: 

a) projecting a grid onto the module; the grid being located 
at a first position relative to the camera and to the module; 



t^rinfteci:21-05^2001 ; |i 



6 



b) taking, with the camera, an image of the module 
illuminated by the projected grid; the image of the module having intensity 
values for each pixel; 

c) repeating steps a) and b) at least two times with the grid 
5 being located at two different known positions relative to the camera and 

to the module to yield at least three intensity values for each pixel; 

d) computing the module phase for each pixel using the at 
least three module intensity values for the corresponding pixel; 

e) computing a complementary phase of the substrate by 
10 using the at least three intensity values from pixels on the image of the 

module not corresponding to the at least one object; 

f) computing the height of the at least one object for each 
pixel using the complementary phase of the substrate and the module 
phase for the corresponding pixel; 

15 g) projecting the grid onto a reference plane; the grid being 

located at the first position; 

h) taking with the camera an image of the reference plane 

illuminated by the projected grid; the image of the reference plane having 

intensity values for each pixel position; 
20 i) repeating steps g) and h) at least two times with the grid 

being located at the two different positions to yield at least three intensity 

values for each pixel; 

j) computing the reference plane phase for each pixel 

position using the at least three reference plane intensity values for the 
25 corresponding pixel; 
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k) computing the height of the substrate for each pixel 
using the complementary plane phase of the substrate and the reference 
plane phase for the corresponding pixel; and 

computing the height of the module by adding the height of 
5 the substrate and the height of the at least one object. 

According to another aspect of the present invention, there 
is also provided a system for measuring the relief of an object, the system 
comprising: 

10 a grid projecting assembly; the grid projecting assembly 

including a grid, an illuminating assembly that includes a light source to be 
projected through the grid onto the object and a projector to project the 
grid onto the object; the grid being mounted to a support; 

an image acquisition apparatus including a camera provided 
1 5 with an array of pixels; 

a computer configured for 

a) positioning the grid relative to the object and to the 
image acquisition apparatus; 

b) receiving from the image acquisition apparatus at least 
20 three images of the projected grid onto the object and at least three 

images of the projected grid onto a reference object; each of the images 
of the projected grid onto the object corresponding to a different known 
position of the grid; each of the images of the projected grid onto the 
reference object corresponding to one of the known positions of the grid; 
25 c) computing the reference object phase for each pixel 

using the at least three reference object intensity values for the 
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corresponding pixel; 

d) computing the object phase for each pixel using the at 
least three object intensity values for the corresponding pixel; and 

e) computing the difference of height between the object 
5 and the reference object for each pixel using the reference object phase 

and the object phase for the corresponding pixel. 

Other objects, advantages and features of the present 
invention will become more apparent upon reading the following non- 
1 D restrictive description of preferred embodiments thereof, given by way of 
example only, with reference to the accompanying drawing 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 In the appended drawings: 

Figure 1 is a schematic view of a system for inspecting 
the surface of an object according to an embodiment of the present 
invention; 

20 

Figure 2 is a schematic view of both the image 
acquisition apparatus and the grid projection assembly of Figure 1 ; 



25 



%SfZEIT 17. MAI. 



A IIOH DUP V07 C TT n mat a* 



^-05-2001- E-PC17GA00/00834 _ '* CLMS - 



27 



WHAT IS CLAIMED IS: 

1 . A method for measuring the relief of an object using a 
camera provided with an array of pixels, said method comprising: 
5 a) projecting a grid on a reference object; the grid being 

located at a first position relative to the camera and to the reference 
object: 

b) taking, with the camera, an image of the reference object 
Illuminated by said projected grid; said image of the reference object 

1 0 having intensity values for each pixel; 

c) repeating steps a) and b) at least two times with the grid 
being located at two different known positions relative to the camera and 
to the reference object to yield at least three intensity values for each 
pixel; 

15 d) computing the reference object phase for each pixel using 

the at least three reference object intensity values for the corresponding 
pixel; 

e) projecting the grid on the object; the grid being located at 
said first position; 

20 f) taking with the camera an image of the object illuminated 

by said projected grid; said image of the object having intensity values for 
each pixel position; 

g) repeating steps e) and f) at least two times with the grid 
being located at said two different positions to yield at least three intensity 

25 values for each pixel; 

h) computing the object phase for each pixel position using 
the at least three object intensity values for the corresponding pixel; 
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i) computing the difference of height between the object 
and the reference object for each pixel using said reference object phase 
and said object phase for the corresponding pixel; and 

j) using said difference of heights between the object and 
5 the reference object for each said pixel to determine the relief of the 
object 

2. A method as recited in claim 1 . wherein, in at least one 
of steps d) and h), the phase A$ is computed for each pixel by solving 

1 0 the following system of equations: 

I„ = A + B • cos(A0 1- Ap e ) 

where J n represent the at least three intensity values, A and B are known 
coefficients and A<p n are phase variations caused by the different 
locations of the grid. 

15 

3. A method as recited in claim 1, wherein, in step c). steps 
a) and b) are repeated more than two times with the grid being located at 
more than two different known positions relative to the camera and to the 
reference object to yield more than three intensity values and. in step d), 

20 only the three most advantageous values from said more than three 
intensity values are selected and used to compute the reference object 
phase for each pixel. 

4. A method as recited in claim 1 . wherein, in step g), steps 
25 a) and b) are repeated more than two times with the grid being located at 

more than two different known positions relative to the camera and to the 
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object to yield more than three intensity values and, in step d) only the 
three most advantageous values form said more than three intensity 
values are selected and used to compute the object phase for each pixel. 

5 5. A method as recited in claim 1 , wherein, in step c). said 

different known positions of the grid are chosen so as to provide in step 
g) at least two images of the object having a 1 80 degrees difference in 
phase therebetween. 

10 6. A method as recited in claim 5, wherein a two- 

dimensional image of the object is computed by subtracting said at least 
two images of the object having a 180 degrees difference in phase 
therebetween; said two dimensional image being used to perform a 
preliminary analysis of the object. 



15 



20 



7. A method as recited in claim 1 , wherein, in step g), said 
different known positions of the grid are chosen so as to provide at least 
two images of the reference object having a 180 degrees difference in 
phase therebetween. 



8. A method as recited in claim 7, wherein a two- 
dimensional image of the reference object is computed by subtracting said 
at least two images of the reference object having a 180 degrees 
difference in phase therebetween; said two dimensional image being used 
25 to perform a preliminary analysis of the reference object. 
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9. A method as recited in claim 1 , wherein said reference 
object is a plane surface. 

10. A method as recited in claim 1 , wherein said reference 
5 object is said object at a past predetermined lime and said reference 

object phase is computed around said past time; 

whereby step i) provides the variation of height at each pixel between said 
past time and the approximate time when the object phase is computed; 
and step j) yield the variation with time of relief of the object. 

10 

1 1 . A method as recited of claim 1 , wherein said reference 
object is a CAD of the object; said grid being virtually positioned and 

^ projected into said CAD in step a) and said image of said reference object 
being simulated in step b). 

15 

12. A method for measuring the height of a module including 
at least one object mounted on a generally plane substrate using a 
camera provided with an array of pixels, said method comprising: 

I) projecting a grid onto the module; the grid being located 
20 at a first position relative to the camera and to the module. 

m) taking, with the camera, an image of the module 
illuminated by said projected grid; said image of the module having 
intensity values for each pixel; 

n) repeating steps a) and b) at least two times with the grid 
25 being located at two different known positions relative to the camera and 
to the module to yield at least three intensity values for each pixel; 
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o) computing the module phase for each pixel using the at 
least three module intensity values for the corresponding pixel; 

p) computing a complementary phase of the substrate by 
using said at least three intensity values from pixels on the image of the 
5 module not corresponding to said at least one object; 

q) computing the height of said at least one object for each 
pixel using said complementary phase of the substrate and said module 
phase for the corresponding pixel; 

r) projecting the grid onto a reference plane; the grid being 
1 0 located at said first position; 

s) taking with the camera an Image of the reference plane 
illuminated by said projected grid; said image of the reference plane 
having intensity values for each pixel position; 

t) repeating steps g) and h) at least two times with the grid 
-15 being located at said two different positions to yield at least three intensity 
values for each pixel; 

u) computing the reference plane phase for each pixel 
position using the at least three reference plane intensity values for the 
corresponding pixel; 
20 v) computing the height of the substrate for each pixel 

using said complementary plane phase of the substrate and said 
reference plane phase for the corresponding pixel; and 

w) computing the height of the module by adding the height 
of the substrate and the height of said at least one object 

25 



13. The use of the method of claim 12 for lead-coplanarity 

inspection. 
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14. A system for measuring the relief of an object, said 
system comprising: 

a grid projecting assembly; said grid projecting assembly 
5 including a grid, an illuminating assembly that includes a light source to be 
projected through said grid onto the object and a projector to project the 
grid onto the object; said grid being mounted to a support; 

an image acquisition apparatus including a camera provided 
with an array of pixels; 
10 a computer configured for 

a) positioning said grid relative to said object and to said 
image acquisition apparatus; 

b) receiving from the image acquisition apparatus at least 
, three images of the projected grid onto the object and at least three 

1 5 images of the projected grid onto a reference object; each of said images 
of the projected grid onto the object corresponding to a different known 
position of the grid; each of said images of the projected grid onto the 
reference object corresponding to one of said known positions of the grid; 

c) computing the reference object phase for each pixel 
20 using the at least three reference object intensity values for the 

corresponding pixel; 
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d) computing the object phase for each pixel using the at 
least three object intensity values for the corresponding pixel; and 

e) computing the difference of height between the object 
and the reference object for each pixel using said reference object phase 

5 and said object phase for the corresponding pixel. 
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(57) Abstract: A method and a system for measuring the relief of an object are described herein. The system includes a grid project- 
£2 m 8 f OT projecting a grid, an image acquisition apparatus that includes a camera, and a computer. Providing a reference object having 
g£ common elements with the object to measure, the method includes the steps of a) positioning the grid at three different known posi- 
tions relative to the camera and the common elements; b) for each position of die grid, projecting the grid unto the reference object 
and, with the camera, taking an image of the reference object to yield three images having values for each pixel of the camera and 
c) computing the reference object phase for each pixel using the three reference object intensity values for the corresponding pixel. 
© Steps a), b) and c) are repeated by replacing the reference object by the object to be measured. The difference of height between the 
object to be measured and the reference object for each pixel are then computed by subtracting the reference object phase and the 
object phase for the corresponding pixel. 
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TITLE OF THE INVENTION 

METHOD AND SYSTEM FOR MEASURING THE 
RELIEF OF AN OBJECT 

5 

FIELD OF THE INVENTION 

The present invention generally relates to methods for 
measuring the relief of an object. More specifically, the present invention 
10 is concerned with the use of such systems and methods to inspect the 
lead coplanarity on circuit board. 

BACKGROUND OF THE INVENTION 

1 5 The use of interferometric methods to inspect the surface 

of an object for defects or to measure the relief of an object is well known. 
Generally stated, these methods consist in generating an interferometric 
pattern on the surface of the object and then analyzing the resulting 
interferometric image (or interferogram) to obtain the relief of the object. 

20 The interferometric image generally includes a series of black and white 
fringes. 

Interferometric methods that require the use of a laser to 
generate the interferometric pattern are called "classic interferometric 
25 methods". In such classic methods, the wavelength of the laser and the 
configuration of the measuring assembly generally determine the period 
of the resulting interferogram. Classic interferometry methods are 
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generally used in the visible spectrum to measure height variations in the 
order of micron. 

However, it has been found difficult to use such method 
5 to measure height variations (relief) on a surface showing variations 
beyond 0.5 -1 pm when they are implemented in the visible spectrum. 
Indeed, the density of the black and white fringes of the resulting 
interferogram increases, causing its analysis to be tedious. 

1 0 Another drawback of classic interferometric methods is 

that they require measuring assemblies that are particularly sensitive to 
noise and vibrations. 

Surface inspection methods based on Moire 
15 interferometry allow measuring the relief of an object in the visible 
spectrum with accuracy much more than the accuracy of classic 
interferometric methods. These methods are based on the analysis of the 
frequency beats obtained between 1) a grid positioned over the object to 
be measured and its shadow on the object ("Shadow Moire Techniques") 
20 or 2) the projection of a grid on the object and another grid positioned 
between the object and the camera that is used to take a picture of the 
resulting interferogram ("Projected Moire Techniques"). In both cases, the 
frequency beats between two grids produce the fringes of the resulting 
interferogram. 

25 

More specifically, the Shadow Moire technique includes 
the steps of positioning a grid near the object to be measured, providing 
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illumination from a first angle from the plane of the object (for example 45 
degrees) and using a camera, positioned at a second angle (for example 
90 degrees from the plane of the object), to take pictures of the 
interferogram. 

5 

Since the distance between the grid and the object 
varies, this variation of height produces a variation in the pattern of the 
interferogram. This variation in the pattern can then be analysed to obtain 
the relief of the object. 

10 

A drawback to the use of a Shadow Moire technique for 
measuring the relief of an object is that the grid must be positioned very 
close to the object in order to yield accurate results, causing restrictions 
in the set-up of the measuring assembly. 

15 

The Projected Moire technique is very similar to the 
Shadow Moire technique since the grid, positioned between the camera 
and the object, has a function similar to the shadow of the grid in the 
Shadow Moire technique. However, a drawback of the Projected Moire 
20 technique is that it involves many adjustments and therefore creates more 
risk of inaccuracy in the results since it requires the positioning and 
tracking of two grids. Furthermore, the second grid tend to obscure the 
camera, preventing it from being used simultaneously to take other 
measurements. 



25 
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A method and a system to measure the relief of an object 
free of the above-mentioned drawbacks of the prior-art are thus 
desirable. 

5 OBJECTS OF THE INVENTION 

An object of the present invention is therefore to provide 
an improved method and system for measuring the relief of an object. 

10 Another object of the invention is to provide such a 

system suitable for lead coplanarity inspection. 

SUMMARY OF THE INVENTION 

15 More specifically, in accordance with the present invention, 

there is provided a method for measuring the relief of an object using a 
camera provided with an array of pixels, the method comprising: 

a) projecting a grid on a reference object; the grid being 
located at a first position relative to the camera and to the reference 

20 object; 

b) taking, with the camera, an image of the reference object 
illuminated by the projected grid; the image of the reference object having 
intensity values for each pixel; 

c) repeating steps a) and b) at least two times with the grid 
25 being located at two different known positions relative to the camera and 

to the reference object to yield at least three intensity values for each 
pixel; 
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d) computing the reference object phase for each pixel 
using the at least three reference object intensity values for the 
corresponding pixel; 

e) projecting the grid on the object; the grid being located 
5 at the first position; 

f) taking with the camera an image of the object 
illuminated by the projected grid; the image of the object having intensity 
values for each pixel position; 

g) repeating steps e) and f) at least two times with the grid 
10 being located at the two different positions to yield at least three intensity 

values for each pixel; 

h) computing the object phase for each pixel position using 
the at least three object intensity values for the corresponding pixel; and 

i) computing the difference of height between the object 
15 and the reference object for each pixel using the reference object phase 

and the object phase for the corresponding pixel. 

According to another aspect of the present invention, there 
is provided a system for measuring the relief of an object, the system 
20 comprising: 

a grid projecting assembly; 

an image acquisition apparatus including a camera provided 
with an array of pixels; 

a computer configured for 
25 a) receiving from the image acquisition apparatus at least 

three images of the projected grid onto the object and at least three 
images of the projected grid onto the reference object; each of the images 



WO 01/06210 



PCT/CA00/00834 



6 

of the projected grid onto the object corresponding to a different known 
position of the grid; each of the images of the projected grid onto the 
reference object corresponding to one of the known positions of the grid; 

b) computing the reference object phase for each pixel 
5 using the at least three reference object intensity values for the 

corresponding pixel; 

c) computing the object phase for each pixel using the at 
least three object intensity values for the corresponding pixel; and 

d) computing the difference of height between the object 
10 and the reference object for each pixel using the reference object phase 

and the object phase for the corresponding pixel. 

Other objects, advantages and features of the present 
invention will become more apparent upon reading the following nonr 
15 restrictive description of preferred embodiments thereof, given by way of 
example only, with reference to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 In the appended drawings: 

Figure 1 is a schematic view of a system for inspecting 
the surface of an object according to an embodiment of the present 
invention; 

25 

Figure 2 is a schematic view of both the image 
acquisition apparatus and the grid projection assembly of Figure 1; 
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Figure 3 is a schematic view illustrating the projection of 
a grid on an object; 



the relief of an object according to an embodiment of the present 
invention; 

Figure 5 is an image of a sphere mounted to a board, as 
10 taken by the system of Figure 1 ; 

Figure 6 is an image of the board of Figure 5, illuminated 

by the grid; 

15 Figure 7 is an image computed by the system of Figure 

1 , representing the phase of the board of Figure 6; 

Figure 8 is an image of the sphere of Figure 5 mounted 
to the board, illuminated by the grid; 

20 

Figure 9 is an image computed by the system of Figure 
1 , representing the phase of the sphere with the board of Figure 8; 

Figure 10 is an image illustrating the phase variation 
25 between the images of Figures 7 and 9; 



5 



Figure 4 is a block diagram of a method for measuring 



Figure 11 is an image representing the phase variation 
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between a module comprising lead balls on a substrate and a reference 
surface; 

Figure 12 is an image representing the phase of the 
5 module of Figure 1 1 ; 

Figure 13 is an image representing the phase variation 
between the phase of the image of Figure 12 and the phase image of a 
complementary surface; 

10 

Figure 14 is an image representing the phase variation 
between the phases of the images of the complementary surface and the 
reference plane; 

15 Figure 15 is the image of Figure 14 after unwrapping. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

Turning now to Figures 1 and 2 of the appended 
20 drawings, a system 10 for measuring the relief of an object, according to 
an embodiment of the present invention, will be described. 

The surface inspection system 10 comprises a grid 
projecting assembly 11, an image acquisition apparatus 12, and a 
25 computer 14 advantageously provided with a storing device 16, an output 
device 18 and an input device 20. 
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Turning now more specifically to Figure 2 of the 
appended drawings, the grid projecting assembly 11 and the image 
acquisition apparatus 12 will be described in more detail. 

5 The grid projection assembly 1 1 includes an illuminating 

assembly 22, a grid 24 mounted to a movable support 26 and a projector 
28. 

The illuminating assembly 22 advantageously includes 
10 a source of white light 34 that is projected through the grid 24. For 
example, the source 34 is the end of an optical fiber (not shown) providing 
light from a white light source (not shown). An aspherical lens 36 or any 
other condenser is also advantageously used between the source 34 and 
the grid 24. Other light sources may also be used. It is also believed to 
15 be within the reach of a person skilled in the art to conceive another 
illuminating assembly within the spirit of the present invention. 

The configuration of the grid 24 may vary depending on 
the resolution that is required to adequately measure the relief of the 
20 object 30. For example, it has been found that a ronchi ruling having 250 
lines per inch allows to measure lead coplanarity of a circuit board, where 
a resolution around 1 mm is required. 

The grid 24 is advantageously mounted to a moveable 
25 support 26 that allows displacement of the grid 24 in a direction 
perpendicular (see double arrow 40 on Figure 2) to both the lines on the 
grid 24 and to the direction of incidence of the light (dashed line 42 on 
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Figure 2). 

The movable support 26 is actuated by a stepping motor 
(not shown). The stepping motor is advantageously controlled by a micro- 
5 controller (not shown) triggered by the computer 14. Of course, the 
stepping motor could be directly controlled by the computer 14. 

A projector 28, in the form of a 50 mm TV lens, is 
advantageously used to project the grid 24 onto the object 38. 

10 

The angle 9 between the direction of incidence of the 
light (dashed line 42 on Figure 2) and the line of sight of the image 
acquisition apparatus 12 (dashed line 44 on Figure 2) may vary depending 
on the nature of the object 30 to be measured. 

15 

It is believed to be within the reach of a person skilled in 
the art to position the illuminating assembly 22, the grid 24 and the grid 
projector 28 relative to the object 30 to yield a projected grid having the 
desired pitch p onto the object 30. 

20 

For example, a ronchi grid, having a density of 250 lines 
per inch, with a distance 43 of 22 cm between the object 30 and the 
projector 28, and for an angle 9 of 30 degrees, provides a projected grid 
having a 0.5 mm pitch p. Such a pitch is equivalent to a variation of height 
25 of about 1 mm on the surface of the object 30. 

Obviously, the pitch of the projected grid will vary with the 
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pitch of the grid 24. 

As will be explained hereinbelow, the displacement of 
the projected grid 24 on the object 30 may alternatively be achieved by 
5 fixing the position of the grid 24 and by moving the object 30 and the 
camera 46 together. 

It is to be noted that the system 10 does not require a 
grid between the camera 46 and the object 30. This advantage will be 
10 discussed hereinbelow. 

The image acquisition apparatus 12 includes a camera 
46, provided with an array of pixels, and is advantageously in the form of 
a CCD camera 46. Such a camera provides, for example, a resolution of 
15 1300x1024 pixels. 

The image acquisition apparatus 12 also advantageously 
includes a telecentric lens 48, advantageously mounted to the camera 46 
via an optional extension tube 50. 

20 

The configuration of the image acquisition apparatus 12 
and the distance between the apparatus 12 and the object 30 determines 
the field of view of the image acquisition apparatus 12. Alternatively, a 
desired field of view can be achieved without the extension tube 50 by 
25 distancing the camera 46 from the object 30. 

The CCD camera can be replaced by a conventional 
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camera when the computer 14 is configured to digitize the acquired 
images. 

The computer 14 is advantageously configured to control 
5 the displacement of the grid 24, to process the images of the object 30 
taken by the camera 46 and to analyze these images to measure the relief 
of the object 30. 

The computer 14 is advantageously provided with 
10 memory means allowing storing of the images when they are processed 
by the computer 14 and therefore increasing the processing speed. 

The storing device 16 can be, for example, a hard drive, 
a writable CD-ROM drive or other well-known data storing means. It can 

15 be directly connected to the computer 14, or remotely connected via a 
computer network such as the Internet According to an embodiment of 
the invention, the storing device 16 is used to store both the images taken 
by the image acquisition apparatus 12, the relief of the object 30 and other 
intermediary results. Those files can be stored in any format and 

20 resolution that can be read by the computer 14. 

The output device 20 allows visualization of the images 
and of the data produced by the computer 14, and can take many forms 
from a display monitor to a printing device. 

25 

The input device 18 can be a conventional mouse, a 
keyboard or any other well-known input device or combination thereof 
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which allows inputting of data and commands into the computer 14. 

The storing device 16, the display monitor 18 and the 
input device 20 are all connected to the computer 12 via standard 
5 connection means, such as data cables. 

The computer 14 can be a conventional personal 
computer or any other data processing machine that includes a processor, 
a memory and input/output ports (not shown). The input/output ports may 
1 0 include network connectivity to transfer the images to and from the storing 
device 16. 

Of course, the computer 12 runs software that embodies 
the method of the present invention thereof, as will be described 
15 hereinbelow. 

It is to be noted that the system 10 includes adjustable 
support means (not shown) to position the image acquisition apparatus 12 
and the grid projecting assembly 1 1 relative to each other and to the 
20 object 30. Alternatively, other registration means can be used without 
departing from the nature and spirit of the present invention. 

Before giving a detail description of a method for 
measuring the relief of an object according to an embodiment of the 
25 present invention, the general theory underlying such a method will first 
be described. Since this theory is believed to be well known in the art and 
for concision purposes, it will only be briefly described herein. 
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The intensity l(x,y) for every pixel (x,y) on an 

interferometric 

image may be described by the following equation: 

/ (x, y) = A(x, y) + B(x, y) ■ cos^>(x, y)) (1 ) 

10 where A<P is the phase variation (or phase modulation), and A and B are 
coefficient that can be computed for every pixel. 

Knowing the phase variation A&, the object height 
distribution (the relief) at every point h(x,y) relative to a reference surface 
15 can be computed using the following equation (see Figure 3): 

2/r ■ tan(0) 

where p is the grid pitch and e is the projection angle, as described 
hereinabove. 



25 Although the above equation is valid for a parallel 

projection of the grid on the object, as illustrated in Figure 3 (note that the 
incidence ray 60 from the grid projection are parallel), it is believed to be 
within the reach of a person skilled in the art to use another equation rf the 
grid projection is not parallel. 

30 

For example, it has been found with a pinhole projection 
that the pitch p and the angle e increase with the distance from the grid on 
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the plan of the reference surface (see x on Figure 3). It has been found 
that with a first order approximation, variations in p and e cancel each 
other out and the Equation 2 remains valid within a certain limit of the 
parameters. 

5 

It is believed within the reach of someone skilled in the 
art to re-evaluate the relation between the variation of height h(x,y) and 
the phase A<P, and to make corrections to the relation according to the 
configuration of the system used to measure the relief. 

10 

Turning now to Figure 4 of the appended drawings, a 
method for measuring the relief of an object according to an embodiment 
of the present invention will be described in more detail. 

15 Generally stated, the method consists in measuring the 

relief of an object 30 using the system 10 by performing the following 
steps: 

1 00 - positioning the grid 24 at a first position relative to 
20 a reference object; 

1 02 - projecting the grid 24 on the reference object; 
104- taking, with the camera 46, an image of the 
reference object to gather an intensity value for each pixel of the image; 

1 06 - repeating steps 1 00 to 1 04 at least two times with 
25 the grid positioned at two new different known positions to yield at least 
three intensity values for each pixel; 

108- computing the phase for each pixel using the 
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three intensity values; 

110- repeating steps 100 to 108 by replacing the 
reference object with the object 30 to be measured; 

1 1 2 - computing, for each pixel, the difference of height 
5 between the object 30 and the reference object by using the respecting 
phases thereof for every pixel; and 

1 14 - determining the relief of the object for each pixel 
using the difference of height at every pixel. 

10 These general steps will now be further described with 

reference to a first example where the object 62 to measure is a sphere 
64 mounted to a board 66. An image of said object 62 can be seen in 
Figure 5. 

15 By choosing a similar board as the reference object, the 

difference of height between the object 62 and the reference object will 
provide the height of the sphere 64. The common element to the object 
62 and the reference object is, in this example, the board 66. 

20 In step 100, the grid 24 is moved to a first predetermined 

position using the support 26 that is actuated by the stepping motor. As 
it has been discussed hereinabove, the system 10 includes means to 
register and fix the position of the grid 24 and the camera 46 relative to the 
reference object (and later the object). 

25 

In step 102, the grid 24 is then projected onto the 

reference object. 
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In step 104, the camera 46 takes an image of the 

reference object. 

5 The image includes an intensity value for each pixel of 

the image. The computer 14 stores these intensity values for future 
processing. 

Steps 100 to 104 are then repeated at least twice with 
10 the grid positioned at two new known different positions (step 106). This 
will provide three slightly different images and therefore the three intensity 
values for each pixel. One of the three images of the board illuminated by 
the grid 24 can be seen in Figure 6. 

15 Since Equation 1 comprises three unknowns, that is A, 

B and A&, three intensity values /„ l 2 and l 3 for each pixel, and therefore 
three images, are required to compute the phase variation A<P. 

The two new images are taken following small 
20 translations of the grid 24 relative to the surface of the reference object. 
The displacements are so chosen as to yield phase variations in the 
images Acp lr A<p 2 and A<p 3 . This results in three equations similar to 
Equation 1 for each pixel of the pixel array of the camera 46: 



25 

with n=1,3. 
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By solving the system of Equation 3, one obtains the 
value of A0. The displacements of the grid 24 are chosen so as to 
advantageously provide different values of A<p x , Aq> 2 and A<p 3 . 

5 

According to a preferred embodiment of the present 
invention, more than three images are taken. This yield additional 
intensity values that can be used to increase the precision of the 
calculated phase. 

10 

Methods according to the prior-art often require the use 
of four images and all four values from these images are taken for phase 
estimation. Since a method according to the present invention requires 
only three images, additional images may be used to increase the 
15 precision and reliability of the method. 

By keeping, for example, four (or more) images, it is 
possible to discard noisy pixels or images and to keep only the pixels 
having the most advantageous intensity values. Indeed, rf one of the four 
20 intensity values is noisy (that can be caused, for example, by an image 
saturation), the corresponding intensity can be eliminated without 
compromising the precision of the resulting phase for this particular pixel. 

Alternatively, more then three intensity values can be 
25 used to traditionally compute the phase using a numerical method such 
as a least square fit. However, such a method could not prevent 
erroneous phase values to be computed for certain pixels, potentially 
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causing imprecision in the computation of the relief of the object. 

According to another preferred embodiment of the 
present invention, the displacements of the grid between the second and 
5 third images (and the fourth image) are chosen so as to provide two 
images having 180 degrees phase variations Aq> n (see Equation 3). This 
allows obtaining an image of the reference object (or of the object) without 
the projected grid. This can be achieved by adding the intensity values of 
the two images phase shifted by 180 degrees. 

10 

More generally, if the sum of the phase variations of 
some of the three or more images taken by the camera 46 is 360 degrees, 
a corresponding two-dimensional image can be obtained by adding the 
intensity values of these images for each pixel. This recomposed two- 
15 dimensional image does not include the projected grid. This image may 
be used to perform a preliminary analysis of the reference object (or of the 
object) that may speed-up any subsequent analysis that will be performed 
on the image or the values that will result from step 112. 

20 In step 108, the phase is computed using the three 

intensity values (or the three best intensity values) for each pixel by 
solving the Equations 3. This can be achieved by using a conventional 
numerical method, for example. Numerical methods for solving such 
system of equation are believed to be well known in the art and will not be 

25 described herein. 



The resulting phase of the reference object for every 
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pixel is illustrated in Figure 7. 

When the method of Figure 4 is used to inspect a series 
of objects, steps 100 to 108 may be advantageously performed only once 
5 for the reference object before the inspection. This allows the increase of 
the speed of the inspection. 

Steps 100 to 108 are repeated by replacing the 
reference object by the object to measure, i.e. the object 62. 

10 

One of the images of the sphere 64 with the board 66, 
illuminated by the grid 24, can be seen in Figure 8. 

Since there is no difference in performing steps 100 to 
15 108 with the object and with the reference object, and for concision 
purposes, these steps will not be described again by referring to the 
object. 

The resulting phase of the sphere 64, with the board 66, 
20 is illustrated in Figure 9. It is to be noted that the zone 68 in the image of 
Figure 9 is caused by the shadow of the sphere 64. 

In step 112, the difference of height between the object 
30 and the reference object is computed for every pixel, as obtained in 
25 step 108, by subtracting the phase of the reference object from the phase 
of the inspected object. The resulting image is shown in Figure 10. 
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It is to be noted that the phases computed in step 108 
for the object and for the reference object, and illustrated in Figures 7 and 
9, correspond to surface phases relative to an imaginary projection plan. 

5 When a non-parallel projection of the grid 24 is done, 

this imaginary projection plan becomes slightly curved. This is not 
detrimental with the method for measuring the relief of an object according 
to the present invention since both the images of the object and of the 
reference object are taken with the same system 10. 

10 

Since the phases of the object and of the reference 
object at each pixel correspond to the difference of height between the 
object (or the reference object) and the same imaginary projection plane 
(since the same system with the same optical set-up is used), their 
15 subtraction yields the difference of height between the object and the 
reference object. This allows the image acquisition of the object and of 
the reference object to be performed under different illumination. 

In the optional step 114, the relief of the object, i.e. its 
20 height, is determined for each pixel using the difference of height at every 
pixel between the object and the reference object and knowing the 
dimensions of the reference object. 

As will now appear obvious to a person of ordinary skills 
25 in the art, a method according to an embodiment of the present invention 
can be used to measure the difference of height between two objects (one 
being the reference). In this case, step 1 14 is obviously not performed. 
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In some applications, it may be advantageous to use a 
plane surface on which the object to measure will be laid on during 
measurement as the reference object. 

5 

In some applications, it may be advantageous to provide 
the system 10 with a registration system to help position the object and 
the reference object to a known position relative to the camera. Indeed, 
since a comparison between the object and the reference object is 
10 performed for each pixel, a registration system may ensure that 
corresponding points are compared. 

Such registration system may take many forms including 
indicia on plane surface, a stand or a software program implemented in 
1 5 the computer. 

It is to be noted that the images may be first acquired 
and then processed at a future time without departing from the spirit of the 
present invention. 

20 

As will be apparent upon reading the present description, 
a method, according to an embodiment of the present invention, allows 
the measure of the relief of an object using white light. 

25 Although the present invention has been described with 

an example where spherical objects are measured, it allows the inspection 
and measurement of objects having other configurations. 
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The same object may also act as the reference object 
when the system 10 is used to study the variation in time of an object's 
relief. 

5 

Alternatively, the reference object may be replaced by a 
computer model of the object, generated, for example, by a Computer 
Assisted Design (CAD) that would have been virtually positioned 
according to the set-up of the system 10. 

10 

The reference object may also be a similar object having 
defects within acceptable parameters. Hence, the subtraction of the 
phases of the object and of the reference object will set forth the defect of 
the object under inspection. This aspect of the invention is particularly 
15 interesting to inspect the relief of an object having important variations of 
relief. 



Indeed, since the phase values are limited in the range 

0 to 2n the maximum height h 0 that can be detected by most systems of 

20 the prior-art is 

h - £ ( see Equation 2). 

tan(9) 



25 



Usually the unwrapping of phase is done by using a grid 
having a pitch p sufficiently large to ensure that all height variations will be 
in a single-phase order (0 to 2n). 



PCT/CA00/00834 

24 

A drawback to this is the loss of precision that it implies. 
For example, if the object to be measured is tilted according to the image 
acquisition apparatus, the lost of precision may be important. 

5 The following example will illustrate how a method 

according to the present invention allows prevention of the above 
drawback and relates to lead coplanarity inspection on a circuit board. 

Figure 11 is an image showing the relief of a module 69 
10 comprising a plurality of lead balls 70 on a substrate 72. The image of 
Figure 1 1 is obtained by performing steps 1 10 to 1 14 of Figure 4. In this 
example, the object is the module 69 (including the substrate 72 and the 
lead balls 70) and the reference object is a reference plane surface (not 
shown). 

15 

It can be seen in Figure 1 1 , by the variation in the grey 
shade in the image, that the substrate 72 is not parallel to a plane surface. 
Therefore, such image provides less precision in measuring the height of 
the object than rf the substrate would have been plane. Indeed, it is to be 
20 noted that the tilt in the substrate 72 on the image is not caused by the 
system 12, but reflects the actual configuration of the substrate 72. The 
small variation in height of each lead ball 70 may be lost in the overall 
variation in the substrate 72 profile. 
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Although one can conceive a computer algorithm to 
virtually rectify the substrate on the image, such algorithm may add to the 
inspection process time. This can be seen as a drawback when the 
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inspection is performed in real-time on a production line. 

The proposed solution is to use an approximation of the 
surface of the substrate as a second reference object. 

5 

Indeed, it may be advantageous, at each pixel, to first 
find the height of the substrate 72 relative to a plane surface, secondly the 
height of the lead balls 70 relative to the substrate 72 and to finally add 
these two heights to provide the overall height of the object, i.e. the 
10 substrate with the balls. 

The phase of the module is illustrated in Figure 12 and 
is obtained through steps 100 to 108 of the method of Figure 4. 

1 5 Information about the surface of the substrate 72 is then 

obtained by analyzing the pixel corresponding to the substrate 72 
(between the balls 70) on the image of Figure 12 where a pseudo-phase 
image of a complementary surface is computed. 

20 The height of the balls 70 is computed for each pixel 

(step 112) by subtracting the phase of the module (Figure 12), and the 
phase of the complementary surface. The resulting image can be seen 
in Figure 13. 

25 Similarly, the height of the substrate 72 is computed for 

each pixel (step 112) by subtracting the phase of the complementary 
surface and the phase of the reference plane. The resulting image can be 
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seen in Figure 14. This phase image is then unwrapped (see Figure 15). 

The height of the module 69 is then obtained by adding 
the height of the phases of Figures 13 and 15. 

5 

Although the present invention has been described 
hereinabove by way of preferred embodiments thereof, it can be modified 
without departing from the spirit and nature of the subject invention, as 
defined in the appended claims. 
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WHAT IS CLAIMED IS: 

1 . A method for measuring the relief of an object using a 
camera provided with an array of pixels, said method comprising: 
5 j) projecting a grid on a reference object; the grid being 

located at a first position relative to the camera and to the reference 
object; 

k) taking, with the camera, an image of the reference object 
illuminated by said projected grid; said image of the reference object 
10 having intensity values for each pixel; 

I) repeating steps a) and b) at least two times with the grid 
being located at two different known positions relative to the camera and 
to the reference object to yield at least three intensity values for each 
pixel; 

15 m) computing the reference object phase for each pixel 

using the at least three reference object intensity values for the 
corresponding pixel; 

n) projecting the grid on the object; the grid being located 
at said first position; 
20 o) taking with the camera an image of the object 

illuminated by said projected grid; said image of the object having intensity 
values for each pixel position; 

p) repeating steps e) and f) at least two times with the grid 
being located at said two different positions to yield at least three intensity 
25 values for each pixel; 

q) computing the object phase for each pixel position using 
the at least three object intensity values for the corresponding pixel; and 



WO 01/06210 



PCT/CAOO/00834 



28 

r) computing the difference of height between the object 
and the reference object for each pixel using said reference object phase 
and said object phase for the corresponding pixel. 

5 2. A method as recited in claim 1, further comprising using 

said difference of heights between the object and the reference object for 
each said pixel to determine the relief of the object. 

3. A method as recited in claim 1, wherein, in at least one 
10 of steps d) and h), the phase AO is computed for each pixel by solving 

the following system of equations: 

/„ = A + B- cos( AO + Ap„ ) 

where /„ represent the at least three intensity values, A and B are known 
coefficients and A<p n are phase variations caused by the different 
15 locations of the grid. 

4. A method as recited in claim 3, wherein said system of 
equations is solved using a numerical method. 

20 5. A method as recited in claim 1 , wherein, in step c), steps 

a) and b) are repeated more than two times with the grid being located at 
more than two different known positions relative to the camera and to the 
reference object to yield said at least three intensity values and at least 
one additional value for each pixel and, in step d), a selection is performed 

25 among the at least three intensity values and the at least one additional 
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values to yield the three most advantageous intensity values; said three 
most advantageous intensity values being used to compute the reference 
object phase for each pixel. 

5 6. A method as recited in claim 1, wherein, in step c), steps 

a) and b) are repeated more than two times with the grid being located at 
more than two different known positions relative to the camera and to the 
reference object to yield more than three intensity values and, in step d), 
the three most advantageous values from said more than three most 
10 advantageous intensity values are used to compute the reference object 
phase for each pixel. 

7. A method as recited in claim 1, wherein, in step g), steps 
e) and f) are repeated more than two times with the grid being located at 

15 more than two different known positions relative to the camera and to the 
object to yield said at least three intensity values and at least one 
additional value for each pixel and, in step h), a selection is performed 
among the at least three intensity values and the at least one additional 
values to yield the three most advantageous intensity values and said 

20 three most advantageous intensity values are used to compute the object 
phase for each pixel. 

8. A method as recited in claim 1 , wherein, in step g), steps 
a) and b) are repeated more than two times with the grid being located at 

25 more than two different known positions relative to the camera and to the 
object to yield more than three intensity values and, in step d) the three 
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most advantageous values form said more than three intensity values are 
used to compute the object phase for each pixel. 

9. A method as recited in claim 1, wherein, in step c), said 
5 two known positions of the grid are chosen so as to provide at least two 

images of the object having a 180 degrees difference in phase 
therebetween. 

10. A method as recited in claim 9, wherein a two- 
10 dimensional image of the object is computed by subtracting said at least 

two images of the object having a 180 degrees difference in phase 
therebetween; said two dimensional image being used to perform a 
preliminary analysis of the object. 

15 11 . A method as recited in claim 1 , wherein, in step g), said 

two known positions of the grid are chosen so as to provide at least two 
images of the reference object having a 180 degrees difference in phase 
therebetween. 

20 12. A method as recited in claim 11, wherein a two- 

dimensional image of the reference object is computed by subtracting said 
at least two images of the reference object having a 180 degrees 
difference in phase therebetween; said two dimensional image being used 
to perform a preliminary analysis of the reference object. 

25 

1 3. A method as recited in claim 1 , wherein said reference 
object is a plane surface. 
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14. A method as recited in claim 1, wherein said reference 
object is said object at a past predetermined time and said reference 
object phase is computed around said past time; 

5 whereby step i) provides the variation of height at each pixel between said 
past time and the approximate time when the object phase is computed. 

1 5. A method as recited of claim 1 , wherein said reference 
object is a CAD of the object; said grid being virtually positioned and 

10 projected into said CAD in step a) and said image of said reference object 
being simulated in step b). 

16. A system for measuring the relief of an object, said 
system comprising: 
15 a grid projecting assembly; 

an image acquisition apparatus including a camera provided 
with an array of pixels; 

a computer configured for 

a) receiving from the image acquisition apparatus at least 
20 three images of the projected grid onto the object and at least three 
images of the projected grid onto the reference object; each of said 
images of the projected grid onto the object corresponding to a different 
known position of the grid; each of said images of the projected grid onto 
the reference object corresponding to one of said known positions of the 
25 grid; 
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b) computing the reference object phase for each pixel 
using the at least three reference object intensity values for the 
corresponding pixel; 

c) computing the object phase for each pixel using the at 
5 least three object intensity values for the corresponding pixel; and 

d) computing the difference of height between the object 
and the reference object for each pixel using said reference object phase 
and said object phase for the corresponding pixel. 



10 



17. The use of the method of claim 1 for lead-coplanarity 

inspection. 
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POSITIONING THE GRID AT A FIRST POSITION 
RELATIVE TO A REFERENCE OBJECT 
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102 



PROJECTING THE GRID ON 
THE REFERENCE OBJECT 
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104 



TAKING WITH THE CAMERA AN IMAGE OF 
THE REFERENCE OBJECT TO GATHER AN INTENSITY 
VALUE FOR EACH PIXEL OF THE IMAGE 
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106 



REPEATING STEPS 100 TO 104 AT LEAST TWO 
TIMES WITH THE GRID POSITIONED AT TWO NEW 
DIFFERENT KNOWN POSITIONS TO YIELD AT LEAST 
THREE INTENSITY VALUES FOR EACH PIXEL 



I 



108 



COMPUTING THE PHASE FOR EACH PIXEL 
USING THE THREE INTENSITY VALUES 
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110 



REPEATING STEPS 100 TO 108 BY REPLACING THE ~~ 
REFERENCE OBJECT WITH THE OBJECT TO BE MEASURED 



I 



112 



COMPUTING FOR EACH PIXEL THE DIFFERENCE 

OF HEIGHT BETWEEN THE OBJECT AND THE 
REFERENCE OBJECT BY USING THE RESPECTING 
PHASES THEREOF FOR EVERY PIXEL 



I 



114 



DETERMINING THE RELIEF OF THE OBJECT FOR 
EACH PIXEL USING THE DIFFERENCE 
OF HEIGHT AT EVERY PIXEL 
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